Greg Detre
Quillian, M. (1968) Semantic Memory. In M. Minsky, ed, Semantic Information Processing, 216-270. MIT Press, Cambridge, MA.
Harnad, S. (1990) The Symbol Grounding Problem. Physica D 42: 335-346.
1. Summarize each paper�s main ideas (one page maximum for each summary)
2. Consider a machine that represents word meanings in terms of a
Quillian network. What sort of linguistic processing tasks would be inherently
easy for such a machine? What tasks would be difficult?
3. Are semantic networks compatible with Harnad�s notion of symbol
grounding? If so, how? If not, why not?
Harnad�s tri-partite scheme provides an interesting attempt to reconcile what might broadly be termed the symbolic and connectionist agendas.
On the one hand, we have the Fodorian emphasis on the symbolic, citing high-level human cognitive abilities like language and chess as evidence that we can (in principle) capture all that is important about the way our minds work in purely symbolic terms, that is, in terms of manipulations of tokens (symbols) according to rules (which can themselves be couched in symbolic terms). Harnad defines a symbolic system as one which fulfils the following criteria:
Broadly, he defines a symbol system as:
On the other hand, connectionist systems are a subset of statistical, self-organising dynamical systems, with at least some degree of biological plausibility/inspiration. We can broadly characterise the connectionist as hoping that, given certain architectural constraints, the kind of high-level, stable yet flexible, self-learning cognitive models that people have will emerge out of numerous low-level interactions between very simple nodes.
Harnard weighs the relative attractions and disadvantages of the two approaches to modelling the mind. In short, his main points are that connectionist systems are poor at symbolic rule-following, while symbolic systems suffer from the �symbol grounding problem�. There are also numerous other ways of comparing the two approaches[1], that Harnad briefly considers.
Harnad�s paper tries to break down this symbolic/connectionist opposition by postulating a level of iconic and categorical representations on which the symbolic sits. He seems to want the iconic representation to be a slightly-abstracted sensory/imagistic representation (used for discrimination), while the categorical representation is a feature-invariant description which captures all objects of the same name (i.e. used for identification). Together, the sensory and pattern-recognising representations provide the grounding for the symbolic level.
First of all, it�s unclear to me why there is a need for the distinction between the iconic and the categorical representations that Harnad introduces. A connectionist �categorical representation�, for example, already incorporates robust prototyping/generalising/pattern-matching properties that can perform both the discriminating and identifying tasks. In this way, there is no need to make the hard-and-fast distinction between discrimination (a relative ability) and identification (an absolute ability) components.
More importantly, I think there remains a deep question as to whether there is any need to explicitly build a high-level symbolic level at all. After all, although we certainly can manipulate symbols, this ability varies greatly from person to person, is prone to error, and slow (especially in comparison with the overall computational/processing power of our brains), which I think lends support to the idea that our symbolic manipulation abilities emerge out of lower-level non-semantically evaluable interactions[2]. He fails, I think, to show why there really must be a discontinuity between iconic/categorical representations and the kind of fallible, limited, effortful symbolic manipulation that we can perform. Without the symbolic system at the top, there appears no symbol grounding problem at all. Of course, this just shows up my bias against the language of thought hypothesis in general.
Having said that, I�m not sure that I fully understand why symbolic systems suffer from a symbol grounding problem whereas connectionist systems don�t. The way that Harnad couches the general problem of what it is for a word to have meaning seems to presuppose Searle�s distinction between derived and intrinsic intentionality. We can either attack this distinction[3], or try and argue that Searle is too restrictive in his application of it. For instance, I think he fails to adequately respond to the Robot Reply, when he simply asserts that �the addition of such �perceptual� and �motor� capacities adds nothing by way of understanding�. This leaves him either in Roger Penrose�s[4] camp asserting that the human brain is fundamentally non-algorithmic in its operation, or making the even less defensible claim that there is simply something about biological, terrestrially-evolved neurons that makes them special/conscious/originally-intentional. Am I right in thinking that Harnad uses the term �symbolic grounding� to mean little more than a systematic causal feedback loop from the representations in our head through the environment (mediated by our sensory and motor systems) and back? If this is the case, then another way of seeing the problem is that we do not know whether it is possible to design an algorithmic system (i.e. one whose rules are purely syntactic) which is semantically interpretable, and yet which preserves the isomorphism between the environment and the internal representation. If the problem of symbolic grounding can be stated in this way, in terms of the conflict between a semantically-interpretable algorithm and an isomorphism with the environment, then it doesn�t seem inconceivable to me that a future symbolic system could be symbolically grounded in the way that Harnad demands.
A semantic network has been defined as �a declarative graphic representation that can be used either to represent knowledge or to support automated systems for reasoning about knowledge�[5], and this seems to more or less capture what Quillian is trying to do. Each concept is placed within a tangled hierarchy and connected to a multitude of other related concepts by various types and strengths of links (e.g. grammatical (subject/object/direct object, in/for), contextual/semantic etc.) which serve to propagate activation from one node to another.
I liked the way that he approached the general problem of modelling or reproducing human-like meaning and memory faculties. He emphasised commonalities between words (or better, word-concepts) rather than trying to define concepts in isolation, and he tries to incorporate the natural fuzziness and blurredness that seem so essential to our thinking.
At first sight, Quillian�s emphasis on the interconnectedness of his symbolic concepts as the main way in which meaning builds up seems connectionist in spirit. However, we can see that it departs crucially from what Harnad would term a connectionist model because it fulfils all of his 8 criteria for being a symbol system. Quillian�s model therefore falls on the Fodorian symbolic/language-of-thought side, in (implicitly) making the claim that we don�t need to descend below the symbolic level when modelling the mind[6].
The interesting part of Quillian�s paper concerns how activity propagates between nodes, as evidenced by the task of seeing similarities between concepts. The results he includes seem promising, but we need to consider two major issues: scaling, and task specificity.
I don�t think his model would scale to the whole of human language for two major reasons, and two minor reasons:
1. The biggest problem concerns the task of encoding the information into a form that Quillian�s model can absorb. He doesn�t give much information about this process, other than mentioning the vague hope that as more concepts are added, the process will become increasingly autonomous. I think the opposite is true � as the number of concepts increases, the business of relating new concepts by hand would increasingly labyrinthine, and claiming that beyond some hypothetical critical mass, this problem will go away, seems like wishful thinking. It is also worth considering that because the concepts in human language are so densely-connected, the combinatorial explosion of connections and nodes might well become unmanageable (in terms of computational time and storage space) as the model increased from less than a thousand concepts to a hundred times that number.[7]
2. The second major problem is one of task specificity. By this, I mean that although Quillian�s model does a credible job of seeking similarities between the concepts it knows (the task it seems primarily designed around), I don�t think it would be able to re-deploy its knowledge in a more general way. Most importantly, I don�t think that the knowledge encoded in this way would help much in reading, understanding and responding flexibly to sentences in English. It might help occasionally to choose between possible parse trees of a sentence by disambiguating homonyms, but otherwise I just don�t see how it could be merged with the non-linguistic parts of a complete cognitive system.
Another way of saying this is in terms of the trade-off between representational expressiveness and computational tractability � in my opinion, Quillian�s approach concedes too much to computational tractability. The catalogue of link-types he proposes falls between two stools: a) the list of different types of links he allows is too rigid and impoverished to be able to represent the whole gamut of different relations between concepts, and b) by having categories rather than just scalar synaptic weights as in a connectionist system, his system will not self-organise dynamically. As a result, there will always be things that we can say with language that I don�t think any variation or simple expansion of Quillian�s model would be able to represent.
3. A more minor concern relates to how we learn language as children. It seems possible that a really important part of this process may involve stepping-stone or bootstrapping concepts, which we leave behind as our understanding become more sophisticated but which are necessary childhood steps. Alternatively, most of our concepts� meaning/function changes greatly over time, and it could be that their circuitous evolution is absolutely necessary for the system to reach a given end state. By trying to leapfrog all of this to the desired end-state, we may be making the task considerably harder for ourselves.
4. One further minor consideration relates to whether Quillian intended his model as a model of how human memory actually works, or whether it�s merely a reverse-engineered replication of its salient properties. He does state that he is �disposed to consider this model a psychological theory�. Either way, if we were trying to build a machine to pass the Turing test, I think that we could devise innumerable fiendish ways to distinguish even a successful implementation of Quillian�s model from a real human[8].
Both papers were stimulating, and contained a number of interesting ideas. Although I found quite a lot that I wanted to criticise or at least discuss in them, I felt that both authors� concern with meaning was instructive. Both of their approaches could be broadly summed up by Quillian�s four key assumptions about word concepts stored in memory: �that the information in them is large, differentially accessible, exceedingly rich in expressive power and yet composed of units that represent properties�, which reflects an underlying emphasis on the symbolic that they shared. Having said that, I think Harnad�s preoccupation with grounding, rather than simply the internal relations, reflects more modern insights, and will prove necessary for real progress � although Searle�s Chinese Room argument fails to apply as widely as he�d like, it can certainly be levelled here at Quillian�s model.
[1] e.g. see Minsky (1990), �Symbolic vs connectionist� (available at http://www.ai.mit.edu/~minsky/papers/SymbolicVs.Connectionist.txt)
[2] This is what Hofstadter (1979), G�, Escher, Bach argues in various places, such as in his discussion of the possibility of an ant-hill level intelligence/consciousness emerging out of the low-level behaviour of the non-intelligent/conscious ants.
[3] See Dennett�s papers on intentionality
[4] Penrose (1994), Shadows of the mind
[5] Sowa (2002) (http://www.jfsowa.com/pubs/semnet.htm)
[6] See Smolensky, �On the proper treatment of connectionism� in Behavioral and Brain Sciences (1988), 11(1):1-74. In contrast, he maintains that a sub-symbolic level consisting of non-semantically evaluable constituents or micro-features of symbols exists, above the neural level, at which we will be able to fully specify (i.e. capture nomologically) mental activity.
[7] However, the CYC project (www.cyc.com) has shown that neither the hand-coding on an enormous scale nor the associated combinatorial explosion need be considered insurmountable problems. CYC is an analogous modern project whose knowledge base is couched in terms of propositional axioms rather than the kind of definitional hierarchy that Quillian is constructing, but it has a similar aim of building a symbolic corpus of common-sense knowledge or meaning.
[8] Robert French considers a number of such questions, designed to probe what he terms the sub-cognitive framework of the interviewee � see French (1990), �Subcognition and the limits of the Turing test�, http://www.ulg.ac.be/cogsci/rfrench/turing.pdf